This invention relates generally to a Internet and other network-based image processing and display system and, more particularly, to a method and apparatus for inputting two-dimensional images, converting the image to a three dimensional or sequential view image file with user selectable image parameters and outputting the file on a user selectable display or printer unit.
Various methods and apparatus for forming an image that appears to have three dimensions when viewed are known in the art. The term xe2x80x9cthree dimensions,xe2x80x9d for purposes of this description, is for the image of an object, or arrangement of objects, to have an appearance of height, width and depth. This contrasts with conventional photographs and unmodified digital format conversions of the same, which display the dimensions of height and width but, for reasons including lack of parallax, do not display a true image of depth.
There are at least two known methods for arranging and re-formatting two-dimensional photographic images, or pluralities thereof, onto a flat medium that when viewed create an impression of depth. One is commonly known as the xe2x80x9c3-d glassesxe2x80x9d method. In its simplest form, a scene is photographed with two cameras, one corresponding to a person""s left eye and one corresponding to a person""s right eye. The developed pictures, or sequence of pictures for a movie, taken by the two cameras are then projected, one image atop the other, onto a flat screen through two respective projector lenses. The projector lenses apply a different color or polarization to the left and right image, with respect to one another, before overlaying them on the viewing screen. The viewer then wears special glasses that filter, by color or polarization, the overlaid images such that his or her left eye sees only the image from the xe2x80x9cleft eyexe2x80x9d camera, while his or her right eye sees only the image from the right eye camera. Because of the parallax between the image seen by the left eye and right eye, the viewer senses an apparent depth, i.e., three dimensions.
There are, however, problems with the xe2x80x9c3d glassesxe2x80x9d method. One problem is that the viewer must wear the special glasses. Another is that many viewers become nauseated due to visual distortions when viewing the picture.
A second known method for transforming a two dimensional photograph onto a different medium which displays it to have apparent depth is the lenticular lens method. The lenticular lens method interlaces Q different images or Q viewing angles of a single image, using a raster type interlacing, and then places a sheet formed of a plurality of elongated strip lenses, or lenticules, over the raster image. The overlay is such that each lenticule or lens overlays Q raster lines. The lenticules are formed such that one image is presented to the viewer""s left eye and another image is presented to the viewer""s right eye. The difference between the left image and the right image approximates the parallax that the viewer would have experienced if viewing the original image in person.
The optical principles of lenticular screen imaging are well known to one of ordinary skill in the relevant art. However, referring to FIGS. 1A and 1B, the principles of operation will be described.
Referring to FIG. 1A, a lenticular plastic 2 consists of clear plastic containing a vertical overlay of N cylindrical lenses 4, commonly referred to as xe2x80x9clenticulesxe2x80x9d, on the front surface 2a of the plastic. These lenses 4 image light in one direction and are historically designed with their focal points on the back plane 2b of the plastic. The focal plane of each cylindrical lens 4 is measured from the apex 4a of the single refractive surface and is consequently equal to the overall thickness of the plastic sheet 2. FIG. 1A also shows a top view of a typical viewing situation of a person (not numbered) looking at the plastic sheet 2 through his or right eye R and his or her left eye L. It is assumed for this example that the viewer has the average inter-pupil distance, which is 2.5 inches.
As shown in FIG. 1A, the viewer looks at an image at the vertical centerline VC. For ease of understanding only three of the N lenticules 4 are shown, and each is depicted with a significantly enlarged scale. As shown in FIG. 1A, three rays of light, labeled as a, b, and c, radiate from points under the lenticular sheet 2 labeled A, B, and C, respectively. Point A is under lenticule L1, point B is under lenticule L2 and point C is under lenticule L3. Each of the three rays of light a, b, and c pass through the center of curvature of its respective lenticule L1, L2 and L3 and travels to the right pupil R of the viewer.
The light rays a, b and c are straight lines because they each emerge normal to the cylindrical surface 4a of their respective lenticules and, therefore, are not refracted. Further, as shown at FIG. 1B, each of the light rays emerging from point C other than the center ray c will emerge from the lenticule L3 parallel to c. The off-center rays are parallel to c due to their respective angle of refraction at the L3a surface. Therefore, all rays from points A, B and C will emerge parallel to a, b, and c. In other words, points A, B and C are imaged into infinity since they lie in the focal plane of the three lenticules.
The viewer""s left eye will see points D, E, and F, by way of rays d, e, and f passing through center of the respective center of curvature of the lenticules L1, L2, and L3. As shown at FIG. 1A, the points D, E, and F are displaced horizontally on the surface 2b with respect to the points A, B and C.
All of the remaining lenticules (not shown) have a pair of points such as A and D of lenticule L1, one being visible by the viewer""s right eye and the other being visible by the viewer""s left eye.
Referring to FIG. 1A, the lenticules L1, L2 and L3 are shown in cross-section. Seen from a front view (not shown) each of the N lenticules extends a vertical length equal to the height of the screen 2. The points A and D of FIG. 1A extend the same length along a narrow width. Therefore, each lenticule covers two thin vertical areas, one being visible by the viewer""s right eye and the other being visible by the viewer""s left eye.
When the analysis of lenticules L1, L2 and L3 is expanded to include all N lenticules of the viewed sheet 2, it can be seen that the viewer""s left eye sees one set of N vertical fine areas, one behind each lenticule, with his right eye, and a different set of N vertical fine areas with his left eye. As described above, the left and right vertical fine areas under each lenticule are horizontally displaced with respect to one another.
Referring to FIG. 1B the width of each of the vertical fine areas is a function of the acceptance angle, and the angle subtended by the viewer""s pupil. This width is normally a small fraction of the width WL of the lenticule.
An example of the width of the vertical line areas is as follows:
Assume a lenticular sheet with an acceptance angle of 32 degrees and a viewer with xe2x85x9xe2x80x3 pupils located 17 inches from the sheet, as shown in FIG. 1A. From any given in the sheet, the viewer""s pupil subtends an angle of arctan (0.125/17), which equals approximately 0.42 degrees. Hence, for this example, the viewer sees a line behind each lenticule which is 0.42/32, or 1.3% of the lenticule width.
Thus, if an image is converted into N vertical raster lines, placing one behind each of the lenticules L1, L2 and L3, centered on points A, B, and C, respectively, and each of the remaining N-3 lines on an appropriate vertical line behind each of the remaining lenticules, the image would be visible only through the viewer""s right eye. Similarly, if a second image is converted into N vertical raster lines, with one placed under each lenticule at locations corresponding to points D, E, and F, for lenticules L1, L2 and L3, that image would be visible only to the viewer""s left eye.
If the first and second images are a stereo pair, the first representing a scene viewed from the left eye, and the second representing the scene viewed from the right eye, the viewer perceives the same parallax as experienced by an actual viewer""s eyes. However, if only two vertical thin areas are under each of the lenticules, this three-dimensional effect is obtained only when the viewer is at the position depicted by FIG. 1A. The reason is that if the viewer is displaced in a horizontal direction, i.e., lateral with respect to the centerline VC, then the left eye will not see one of the images simultaneous with the right eye seeing the other. For this reason, typically four or more 2D frames, or views, are recorded under the plastic sheet 2, with a corresponding four or more vertical raster lines behind each lenticule. The four 2D frames, and associated raster lines of each, are positioned such that the viewer has four acceptable viewing angles. At the first angle the viewer would see images 1 and 2. Images 1 and 2 would a first stereo pair. At a second viewing angle the viewer""s right eye would see image 2 and his left eye would see image 3. Images 2 and 3 would be another stereo pair of the original scene, in other words two views of the original image having the same parallax with respect to one another as images 1 and 2. Similarly, at a third viewing angle the viewer would see pictures 3 and 4. The optimum number of pictures or frames is easily computed as follows, using FIG. 1A as an example:
The viewer""s eyes subtend an angle of arctan (2.5/17) which equals approximately 8 degrees. Using lenticular material with an acceptance angle of 32 degrees, {fraction (32/8)} or four frames are seen to be a minimum. At this value the viewer""s eyes would see the center of adjacent image strips. More frames can be recorded, but trade-offs such as sharpness versus roundness are involved.
The four-frame example above described the image pairs 1 and 2, 2 and 3, 3 and 4, and 1 and 4, which at the designated viewing angles are seen by the right eye and left eye, respectively, as stereo pairs of the same original image. As known in the art, other image effects are possible with lenticular screens. One of the is xe2x80x9cflip imagingxe2x80x9d in which, for example, image pair 1 and 2 which the viewer would see from a first viewing angle, are a stereo pair of a first image such as, for example, an airplane with its landing gear retracted. Image pair 3 and 4, which the viewer would see from a second viewing angle, could be an image of the airplane with its landing gear out. Another effect is xe2x80x9cactionxe2x80x9d in which a sequence of two or more images is seen as the viewer moves through a corresponding sequence of viewing angle. An example xe2x80x9cactionxe2x80x9d is a baseball pitcher moving his arm. Another effect is xe2x80x9cmorphingxe2x80x9d in which a sequence of two or more images is seen as the viewer moves through a corresponding sequence of viewing angles. The image sequence portrayed is a staged change in the scene, or in one or more objects or characters in the scene.
The current techniques and related apparatus for transferring a two-dimensional image into a lenticular screen format require considerable time by skilled persons, involving a substantial amount of trial and error, and result in an image far below what is termed xe2x80x9cphotographic quality.xe2x80x9d More particularly, producing a multi-dimensional lenticular image from a series of 2D image frames always involves the process of xe2x80x9cline-formingxe2x80x9d, i.e., to slice each 2D frame sequentially into a series of n thin lines, equally spaced for recording behind each of n lenticules.
Presently there are two techniques utilized to produce line formed lenticular imagery:
1. Line formed from photographic interlacing;
2. Line formed from interlacing computer programs used by specialists in lenticular graphic arts.
In photographic techniques either the client must provide a specialist with multiple negatives or multiple negatives must be reduced from existing digital files. The word xe2x80x9cspecialistxe2x80x9d is used because the level of skill and experience required to carry out the photographic interlacing is very high. There are a very small number of such specialists known in the art. A brief summary of the photographic interlacing operation, as presently practiced in the art, is as follows:
Exposure of each different photographic negative through the lenticular screen produces a line formed image. Exposure onto photographic lenticule (emulsion coated directly onto the rear surface of the lenticular sheet) will produce a single unique image. Exposing each frame sequentially through the front of the lenticular produces a xe2x80x9cfuzzyxe2x80x9d master when the lenticular is held in intimate contact with a photographic transparency material. After exposing the sequence of images, the positive can be digitized to produce files for graphic art reproduction in any number of formats such lithography, digital printing, etc.
Specialized equipment is required to produce photographic imagery. The cost and time required is dependent on the number of images, availability of equipment and materials and the skill of the technician producing the image.
Computer image processing requires the image to be in a digital form. Therefore, as a first step, if the image is in a photographic form it must be digitized. This can be done by the client or by the specialist. Next, the digital format must be transported to a specialist who reformats it into multiple images. These multiple images are then line formed, by a specialist, into a form suitable for affixing to a MOM to present as a multi-dimensional image.
However, there is no system known in the prior art for readily converting, by or under the command of an unskilled person without specialized equipment, a two-dimensional image into a three dimensional lenticular image, regardless of image quality. More specifically, there is no known system that allows an ordinary consumer to input a two-dimensional image and, without a significant amount of skilled effort and costly equipment, obtain a reasonable quality lenticular 3D product.
An object of the present invention is to provide a system and method for inputting a two-dimensional image, either a data file or a scanned photograph, into an Internet access terminal, such as a personal computer, transferring the file over the Internet to a server having a 2D-to-3D or other image transformation method programmed into memory, performing the 2D-to-3D or other image processing, then transferring the 3D or other file to a user-designated output device for fixation on a micro-optical material which, when viewed by the generation as a photographic quality lenticular 3D or other image.
A further object of the present invention is to provide a system and method for transferring a 2D file over the Internet, generating a sequence of image files representing a changing, or morphing, of one or more objects in the image, and then transferring the morph file to a user designated output device for fixation on a MOM media. When the MOM is rotated with respect to the plane of the viewer""s eyes, the viewer sees a morphing image.
A still further object of the invention is to provide a system and method for receiving one or more 2D image files over the Internet and transforming the received files into a set of discrete or xe2x80x9cflipxe2x80x9d images, for output to, and fixation on a MOM medium.
Another object of the invention is to provide a system and method for receiving one or more 2D image files over the Internet and transforming the received files into a sequential set of xe2x80x9cactionxe2x80x9d images, for output to, and fixation on a MOM medium. An example of an xe2x80x9cactionxe2x80x9d image is a baseball pitcher, with a sequence of images corresponding to the pitcher""s arm while pitching a baseball.
A further object is to provide a system and method for transferring interphased image files generated, for example, by the system and method of the previous embodiments, to a cathode ray tube (CRT) having an overlaid MOM screen. In an example embodiment the CRT may be local to a client. Further, the image files may be stored, in either 2D or interphased 3D form, in an archival storage accessible to the client by, for example, the Internet.
One example embodiment of a system of this invention meeting these and other objectives comprises a home computer having Internet or other network access to a remote server, a server having data processing and storage capability or resources, and a printer or other output device accessible by the server, either local to the user, local to the server, or at another location. The server stores a 2D-to-3D image processing program for controlling its processor to perform the described steps. An example embodiment of the inventive method on this system is as follows:
a. The user/client enters or downloads a two-dimensional image file into his or her personal computer, or set-top Internet access unit. The file can be in any image formats such as, for example, JPEG or Adobe(copyright) xe2x80x9c.pdsxe2x80x9d, xe2x80x9c.epsxe2x80x9d or the like. The file may be compressed, using any of the techniques known to one of skill in the art.
b. The user accesses a web site, or the like, posted by an image services provider, clicks on an xe2x80x9cupload 2D filexe2x80x9d button or the like, and the file is automatically transferred over the Internet to a server designated by the image services provider, using any of several known transmission media including, but not limited to 56 Kbit/sec modem, ISDN, DSL, T1, T2, T3 or cable modem.
c. In response to prompts, or other graphical user interface requests for input appearing on the user""s display, the user enters commands, via a key pad, mouse, or touch-screen, into a graphical user interface area of the screen to designate an object as the Key Subject. The user may also identify other objects as foreground or background.
d. a program resident on, or accessible from, the server arranges, or increments, different segments or objects, or portions of objects within the two-dimensional image onto respective surfaces, one being a key object surface, the first being the foreground surface, and the last plane being the background surface.
In the above step (d) each surface corresponds to points within the original two dimensional image which, from the perspective of a person actually observing the three-dimensional scene represented by that image, would be at the same distance from that observer. The number of surfaces is selectable.
Next, the system of this example performs the following steps:
e. A number of viewing angles is selected and then, for each viewing angle, a parallax or image shift for each object within each surface is calculated.
f. For each viewing angle, a parallax or image shift based on the results of step (e) is applied to all objects, the amount of shift depending on which surface the object is on relative to the key subject, and the amount of depth between surfaces.
g. A line file corresponding to each viewing angle is generated, each line file representing an image of all objects as seen from that viewing angle, the number of lines corresponding to the number of cylindrical lenses in a MOM sheet.
h. The line files generated by step (g) are interphased into a single line form file.
i. The line formed file is printed on directly on a MOM, printed on paper which is then overlaid with a MOM, photographically applied to a MOM, or displayed on a cathode ray tube (CRT) having a MOM overlay.
A further embodiment of the invention receives, from the client over the Internet, a sequence of two-dimensional images representing morphing of an object, an action, a flip or a zoom sequence. The sequence of images is interphased into a merged file. For a morph image the sequence of files represents a progressive change in the shape or form of an object. The interphased or merged file is printed or photographically exposed on a MOM. When viewed through the MOM at a succession of viewing angles, presents a corresponding sequence of visual images. Similarly, for action, flip and zoom the MOM, which when viewed through the MOM at a succession of viewing angles, presents a corresponding sequence of visual images appearing as action of an object, a flip between objects or scenes, or a zoom-in or zoom-out from an object.